Skip to content

Say how CI reads the stale-preview class, since it cannot capture the stills - #175

Closed
leaf-agent wants to merge 2 commits into
mainfrom
fix/ci-33443479562
Closed

Say how CI reads the stale-preview class, since it cannot capture the stills#175
leaf-agent wants to merge 2 commits into
mainfrom
fix/ci-33443479562

Conversation

@leaf-agent

Copy link
Copy Markdown
Collaborator

Problem

ci run 33443479562 on 0eec71a came back 3 failed, 1587 passed, 6 skipped. Two are the anchor pair #174 already carries and fixes; the third is test_site.py::test_the_public_catalog_is_a_visual_index_of_full_page_routes — the catalog preview digest, whose handling #168 had just named as its own red-suite class.

I followed that class from CI and it sends a run somewhere it cannot go. "Run the generator and commit a manifest-only refresh" only holds when the stills did not move, and the generator run here cannot tell you that: theme.css asks for Charter, "Bitstream Charter", "Iowan Old Style", Georgia, ui-serif, serif and system-ui for sans, fc-match on the runner has none of the first four, so every capture taken here differs from every checked-in still whether or not the tree changed. A run that follows the sentence literally either commits runner-font stills or bumps inputs_sha256 on faith.

Solution

The class now says which of the two readings it is, and how to get it. Compare the generator's output to itself across two trees — a worktree at the last commit whose manifest equalled its computed digest, and one at the tip — rather than to what is checked in. The substituted faces cancel, so nine identical stills mean only the hash moved and an inputs_sha256-only commit is the whole fix, while any difference means the gallery is genuinely behind and only the authoring machine has the faces to recapture it.

Documentation only; no test or runtime change.

Testing

Ran the A/B for the live gap, which is what the wording is drawn from. b932acb is the last commit on main whose checked-in inputs_sha256 equals digest(capture_input_files()). Capturing there and at b38c6e0, same runner and Chrome: six of the nine stills moved, so this gap lands on the branch that must not be committed from here.

still b932acbb38c6e0
command-hub column moved right 19px
release-notes column moved right 38px, and the opening paragraph was rewritten
design-decision, heat-loss, live-progress, ship-review column moved left 2px
postmortem, pr-walkthrough, triage-board byte-identical

All six moved at bba885a "Canonicalize target Buttons" and nothing moved after it — 0eec71a, bdf9f44 and b38c6e0 each move inputs_sha256 while leaving all nine captures byte-identical. The drift is a content change rather than a rendering artefact: examples/release-notes.html's first paragraph gained "A filled margin Button commits its verb; a hollow one reveals context, and … unfolds more Buttons beside it." and lost "decide the proposed replacement, deletion, and addition in the margin", so https://leaf.page/ is serving a sentence the example no longer has.

pre-commit run --files .claude/skills/running-tend/SKILL.md: pass.

What this PR does not do: clear the red. The digest on main's tip 78fa4c4 computes 1936c559f559e6a710e084dd24c45377bf6ce6d108bd6a032383eb244a46229f against a manifest still reading ff92edc6…, and the six moved stills mean clearing it needs scripts/example-previews.py re-run on the authoring machine, after the branch's own last merge of main. That is the same conclusion as #133, now with the measurement that says a hash-only bump would have been wrong this time — unlike the 03bfc0d7f58225 gap, where all nine came back identical and it would have been right.

How the A/B was run

scripts/example-previews.py run unmodified in a detached worktree per commit on the ubuntu-24.04 runner, same Chrome throughout, comparing sha256 of the nine docs/example-*.jpg between output sets. Trees: b932acb, bba885a, 0eec71a, bdf9f44, b38c6e0; per-still change attributed by comparing consecutive pairs. Horizontal offsets are the shift minimising mean absolute difference over the page area below the banner, at the 896×560 output scale, so ×1.25 for viewport pixels.

digest(capture_input_files()) per tree, against the checked-in inputs_sha256 read with git show <commit>:docs/example-previews.json:

commit manifest computed
b932acb ff92edc6… ff92edc6… matched — the last such tree on main
bba885a ff92edc6… 012f7429… stale, and the only one that moved a still
0eec71a ff92edc6… 09b2364f… reproduces the run's two hashes exactly
bdf9f44 ff92edc6… df040ed3… stale
b38c6e0 ff92edc6… c493c61f… stale
78fa4c4 (tip) ff92edc6… 1936c559… stale

The 0eec71a row reproducing the run's assertion is what says the method matches the gate's.

Fonts, measured rather than recalled — fc-list on the runner holds DejaVu, Lato, Liberation and Noto Color Emoji only; fc-match Charter, fc-match "Bitstream Charter" and fc-match "Iowan Old Style" all return DejaVu Sans, fc-match Georgia returns DejaVu Serif.

Pixel counts for the b932acbb38c6e0 pair, threshold 12/255: command-hub 11.4%, design-decision 10.5%, heat-loss 9.5%, ship-review 8.9%, release-notes 7.4%, live-progress 6.2%, and exactly 0 for the other three.


Automated fix for failed run

@leaf-agent leaf-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things about the A/B as written, both about it returning "only the hash moved" when the gallery has actually moved — the outcome the paragraph exists to prevent.

The substitution that makes the A/B work also blinds it to one input class. theme.css is a capture input (capture_input_files() rglobs skills/leaf/assets), so editing --serif — reordering it, dropping Georgia, adding a face — reddens the test. On the runner every name in that stack resolves to the same substitute (fc-match Charter and fc-match "Iowan Old Style" both return DejaVu Sans here, fc-match Georgia returns DejaVu Serif), so both trees render identically and the A/B says hash-only, while on the authoring machine, where Charter exists, the gallery really did move. Worth the clause, since the font stack is exactly the input a reader of this paragraph is most likely to have touched.

The generator writes docs/example-*.jpg and rewrites the manifest in whichever tree it runs in. "again at the tip" reads as the session's own checkout, which leaves nine runner-font captures and a regenerated manifest as working-tree modifications right before the session is told to make a commit — and the regenerated manifest carries the runner captures' sha256 values, so committing it wholesale trades one red assertion for another. Naming both runs as throwaway worktrees closes that.

Separately, and not something I'd hold the change for: the paragraph names the baseline commit by property but not by how to find it, so the next session re-derives the search — computing digest(capture_input_files()) at candidate trees until one matches its manifest. A clause pointing at the manifest's own history as the candidate list would save that.

The measurements hold up. On the merged tree the manifest reads ff92edc6… against a computed 1936c559…, matching the body's tip row exactly.

Comment thread .claude/skills/running-tend/SKILL.md Outdated
Both generator runs go in throwaway worktrees, so the runner-font stills and the manifest they rewrite never land on the branch. The all-identical verdict now carries the theme.css font-stack exception, and the baseline commit is found from the manifest's own history rather than re-derived.

@leaf-agent leaf-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main moved past this change while the last commit was being written. 4b42d15 "Remove the catalog preview freshness digest" landed at 22:51:04Z, five minutes before 7079fda, and it deletes exactly the four lines this PR rewrites — along with everything the new eighteen lines describe: docs/example-previews.json, capture_input_files()/digest()/inputs_sha256 in scripts/example-previews.py, and the assertion in tests/test_site.py that raised preview inputs changed — rerun scripts/example-previews.py. So the bullet is now keyed on a failure message nothing can emit, and prescribes an A/B over a manifest field that no longer exists. gh pr view 175 --json mergeable reports CONFLICTING for the same reason: the base lines are gone.

The three things the earlier review raised are all handled in 7079fda — throwaway worktrees, the font-stack blind spot, the baseline lookup — so this isn't a quality objection. There is just no longer a class to document: the maintainer's answer to "the digest can't tell CI whether the gallery moved" was to stop coupling the two, which is a level below where this PR was working. Recommend closing.

One measurement in the PR body outlives the mechanism. Six of the nine stills moved at bba885a, and examples/release-notes.html on main now reads "A filled margin Button commits its verb; a hollow one reveals context, and … unfolds more Buttons beside it." while the checked-in still predates that edit — I confirmed the sentence against 4b42d15; the capture dates are this PR’s own A/B. The digest was what flagged that drift, and #133 — which tracked it — closed earlier today, so with the digest gone nothing is watching: https://leaf.page/ is serving a preview of a sentence the example no longer has, and only the authoring machine has the faces to recapture it. Worth a home somewhere other than a PR that closes.

@leaf-agent

Copy link
Copy Markdown
Collaborator Author

Closing: the mechanism this paragraph documents is gone from main. 4b42d15 "Remove the catalog preview freshness digest" deletes the four skill lines this PR rewrites, along with docs/example-previews.json, inputs_sha256/digest()/capture_input_files() in scripts/example-previews.py, and the test_site.py assertion that raised preview inputs changed — rerun scripts/example-previews.py — so the bullet is keyed on a message nothing can emit, and mergeable reads CONFLICTING for the same reason. The review's three points are all in 7079fda and I'm not carrying them anywhere: with the coupling removed there is no class left to document. Leaving the branch in place rather than deleting it.

One verified fact outlives the mechanism, recorded here rather than as an issue since only the authoring machine can act on it: https://leaf.page/ serves a release-notes preview of a sentence the example no longer has. docs/example-release-notes.jpg was last captured at b4a5aeb (2026-08-30 21:53 -0700); bba885a rewrote that page's lede 1h36m later. I read the checked-in still directly — it shows the pre-bba885a lede ("Rewrite any draft in place; decide the proposed replacement, deletion, and addition in the margin…"), while examples/release-notes.html on main reads "A filled margin Button commits its verb; a hollow one reveals context, and … unfolds more Buttons beside it." That is the drift the digest used to flag; nothing flags it now, and a recapture needs the shipped faces.

@leaf-agent leaf-agent closed this Aug 31, 2026
max-sixty added a commit that referenced this pull request Sep 1, 2026
## Problem

[`ci` run
33516407327](https://github.com/max-sixty/leaf/actions/runs/33516407327)
came back **10 failed, 1614 passed, 6 skipped**. Eight belong to #189,
whose `ci` run was cancelled by #190's push before the nightly leg ever
ran on it. #188's `test_mcp_app.py` case arrived on the same collision
as the MCP test below and is carried here rather than left to red the
next nightly.

The everyday suite that gates a pull request does not run
`test_render_*` or `test_site.py`, so a PR can land its nightly half
unread — which is how these arrived, and how five more have arrived
since, from #193, #194 and #200. `main` has moved a long way under this
branch; the ones it now owns are taken on merge, and three of the new
ones are cleared here. #203 reached `tests/conftest.py`'s
`headless_shell` first, so the four `test_render_commands.py` setup
errors are its.

## Solution

Grouped by cause rather than by test, since several tests share one.

**The example the tests still name.** #189 rewrote
`examples/pr-walkthrough.html` into the Worktrunk review packet, and the
shipped patch is now a collapsed manifest that builds no lines until a
reader opens a file, which the scroller gate read as
`getComputedStyle(null)`. The gate reads the diffs that have drawn
lines, with its own non-vacuity floor kept.

**The new package's chrome and surfaces.** `.lf-pr-description` stacked
a second translucent accent onto a card that is already a tint of one,
and the layer's own `--muted` reads 4.32:1 there; the description now
sits on the raised-surface token. CallDiff's disclosure was a bare
`<button>` rather than an `offer`, so an exported copy kept a hand over
a press nothing can take. Its location anchor carried an `href` on the
header row, which names no location at all — and `reachScrollers` reads
a candidate for a focusable descendant before granting it a stop, so
that hidden anchor answered "there is already a way in here" with a link
nobody can reach, for a box whose words run off the side.

**A reading that counted clipped text as painted.** `coveredWords`
measured each run's whole rect, so an ellipsised name in a narrow column
read as covering its neighbour. It now intersects each run with the
boxes that clip it, stopping at an out-of-flow ancestor where a hidden
overflow further out need not reach. The one test that plants this fault
plants it properly now: the runtime's note is parked in a one-pixel box
with hidden overflow, so opacity alone never put its characters on the
screen.

**Three focus rings the corpus stopped painting.** `code-pre-light` and
`code-pre-shadow` are the tab stops on a scrolling code block and on a
diff's lines, and with the old example gone no example held either. The
page gets one rendered hunk and one code line long enough to scroll.
Then `.lf-diff-review`: it took the outward ring it shared with
`.lf-diff-next`, which stands on its own in the toolbar, while this one
rides a file's summary row inside a box that clips at the row's bottom
edge — its lower run fell 3px past what the sweep can see. It takes the
inset the summary beside it already takes, and splitting the rule made
each half declare the ring name neither had.

**`--diffs-fg-number` on changed rows.** Surfaced by that rendered hunk:
a changed row's number is drawn in the base its own fill is mixed from,
green on green at 4.03:1. Pierre's number overrides now take the layer's
tinted-surface inks.

**A sidebar the page got shorter under.** #190's zero-height ToC anchor
shortened the release page 58px, putting a named `scrollTo(0, 900)` past
the stretch where the box stands on its own offset. Both edges of that
stretch are the page's, so they are read off it, with a floor asserting
the stretch exists before a point halfway along it says anything.

**Playwright's loop against `asyncio.run` and `anyio.run`.**
`sync_playwright()` keeps an asyncio loop running in its thread for its
whole lifetime, and the `browser` fixture is session-scoped per xdist
worker — so an MCP test's own loop start raises in any worker that has
opened a browser and passes in one that has not, leaving the schedule to
decide. Verified directly: `asyncio.events._get_running_loop()` is
`None` before `sync_playwright()`, the loop inside it, `None` after.
`interact_support.run_async` runs the entry point on a thread with no
loop on it, and both MCP modules go through it — #204's three new
exchanges included, since they landed on the direct call while this was
in flight.

**A declared verb no page replayed.** #194 added `lf-diff`'s `review` to
the registry without an event for it on the standing-state page — the
one fixture built so that exactly this fails rather than going
unexercised. It gets a diff and a standing review.

**A focus contract two site tests never heard about.** #193 kept the
response field passive so a drag leaves the browser's own selection
alone and a native copy still has something to take, and aligned the
render tests with it. `test_site.py` is nightly-only. Its label test
turned on `document.activeElement === field`, which is now false for the
page's own words and the site's label alike — the discriminator was
gone, not inverted — so it reads whether the field was offered at all,
which is the difference it was always after.

## Testing

`uv run pytest tests --run-nightly` and `pre-commit run --all-files` on
this head. The everyday suite and lint are green on this branch in CI.
Every claim about what `main` does was taken from a control run of the
same tests in a clean checkout of `main`, not from reading.

<details><summary>Where each failure landed</summary>

| test | fix |
| --- | --- |
|
`test_render_gate.py::test_the_runtime_holds_a_scroller_the_page_wrote`
| reads the diffs that drew lines |
|
`test_render_export.py::test_an_exported_example_stands_on_its_own[pr-walkthrough]`,
`[corpus]` | widget, probe, theme |
|
`test_render_controls.py::test_every_ring_the_layer_draws_is_shown_whole_somewhere_in_the_corpus`
| example, then `.lf-diff-review`'s ring and both ring names |
|
`test_render_pages.py::test_a_left_sidebar_uses_the_margin_until_the_page_needs_it_back`
| scroll position read off the page |
|
`test_interact_mcp.py::test_stdio_protocol_carries_the_app_resource_and_private_tool_result`
| own thread |
|
`test_mcp_app.py::test_registered_server_prefers_full_page_and_keeps_snapshot_as_fallback`
| own thread |
|
`test_render_projection.py::test_the_render_gate_applies_every_standing_action_a_second_time`
| standing fixture (#194's) |
| `test_site.py::test_the_label_is_chrome_rather_than_words_to_quote`,
`::test_a_comment_lands_in_the_thread_with_its_quote` | the field is
offered, not entered (#193's) |
|
`test_render_anchors.py::test_a_data_bound_diff_aims_and_selects_one_source_line`
| **#194's and #193's, taken on merge** |
| `test_site.py`, `test_render_export.py` stale `<h1>` | **#202's, taken
on merge** |

The two exported-copy cases each carried four faults, three of them
masked: `offering` asserts before `covered`, which asserts before the
copy's axe run, so each fix uncovered the next. The ring sweep did the
same — the geometry fault stood in front of a naming check that had been
silent about a rule missing its name since it was written.

</details>

<details><summary>Three this leaves, all of them main's</summary>


**`test_render_controls.py::test_examples_have_no_serious_wcag_a_or_aa_violations[pr-walkthrough]`
stays red.** The `.lf-pr-description` contrast above is fixed and
confirmed gone. What holds the test red is 46 `nested-interactive`
findings from #194: each file's "Mark reviewed" `<button>` sits inside
that file's `<summary>`, and a disclosure with a focusable descendant is
a serious WCAG failure. Clearing it means taking the press out of the
disclosure and re-laying the row — `entry.node` is the `<details>` in
ten places, and the theme, print and export rules key on it — which is a
change to that widget's shape with visual verification of its own, not a
line this branch should slip in.


**`test_render_anchors.py::test_a_failed_fragment_hydration_waits_for_a_reader_retry`
fails whenever anything runs before it.** Confirmed on unmodified
`main`: three runs of the file at `-n4`, three failures, and the same
failure at `-n0` over the whole file. It passes alone every time.
`assert len(requests) == 2` sees one — the reopened disclosure's second
fragment fetch never reaches the route. Arrived with #189.


**`test_render_anchors.py::test_a_drag_released_mid_word_hugs_words_and_sentences`
fails over the whole file and passes alone.** `assert 'inside' ==
'inside it'` — the selection did not grow to the word boundary. It looks
like the next instance of what #191 diagnosed, Chromium 151 collapsing a
synthetic range on mouse release.

Also still behind: the gallery's `docs/example-pr-walkthrough.jpg`.
Regenerating here rewrites all nine stills in substituted faces, because
`theme.css` asks for Charter and this runner's `fc-match` has none of
the serif stack — the same reading #175 made. Left for the authoring
machine.

</details>

---

Automated fix for [failed
run](https://github.com/max-sixty/leaf/actions/runs/33516407327)

> _This was written by Claude Code on behalf of @max-sixty_

---------

Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
Co-authored-by: Maximilian Roos <m@maxroos.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant